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Educational Assessment via a Web-Based Intelligent System 



Effective assessment is vital in educational activities. We propose IWAS (intelligent Web-based assessment system), 
an intelligent, generalized and real-time system to assess both learning and teaching. IWAS provides a foundation for 
more efficiency in instructional activities and, ultimately, students' performances. Our contributions are summarized as: 

(1) Given the causes (student knowledge levels and learning styles), BN (Bayesian Networks) technique is utilized to 
automatically reason on the probabilities of the presence of the effects (learning outcomes); (2) The absence of 
teaching practice assessments is addressed via the feedbacks from three different levels, aiming to correlate the 
teaching assessment with the learning assessments for the improved effectiveness in instructional activities; (3) Under 
a client/server architecture, IWAS is decomposed into a set of independent modules; through the standard inter-module 
interfaces, the flexibility of easy maintenance makes IWAS a generalized system adaptable to different domains; and 
(4) Web technologies are integrated to deliver the formative feedbacks to users in a timely manner. 

Keywords: educational assessment, IWAS (intelligent Web-based system), Anderson and Krathwohl’s Taxonomy, 

Felder’ s Model, BN (Bayesian Networks) Reasoning 

Introduction 

Typical educational assessment makes use of preselected measurements, norm-referenced standardized 
tests for example, to measure and evaluate the education quality of students, instructors, classes, institutions 
and the educational system as a whole. Effective and efficient assessment is significant in properly placing 
students, diagnosing problems and progress in learning, improving and enriching teacher’s performance, 
evolving curriculum and achieving and maintaining academic standards. Resnick (as cited in Wiggins, 1990) 
suggested that the role of assessment in education is crucial by stating that “What we assess is what we value. 
We get what we assess, and if we don’t assess it, we wouldn’t get it”. 

Despite the fact that numerous systems have been proposed for different applications of learning and 
teaching assessment, problems have been identified for the existing systems. We summarize these problems as 
follows: (1) overlooking the potential impact of diversified personality and learning styles on learners’ 
performances; (2) students’ learning and instructors’ teaching have not been assessed in a correlated manner in 
order to be more efficient and effective for instructions (Cerbin, 1994); (3) more focuses on specific 
applications and domains, and lacking the generality for broader purposes and fair comparisons; and (4) most 

Jingshan Huang, Ph.D., Assistant Professor, School of Computer and Information Sciences, University of South Alabama. 

Lei He, Ph.D., Research Scientist, National Library of Medicine, National Institutes of Health. 

Gayle V. Davidson-Shivers, Ph.D., Professor, Department of Professional Studies, College of Education, University of South 



Jingshan Huang 
University of South Alabama, 
Alabama, USA 



Lei He 

National Institutes of Health, Maryland, 



USA 



Gayle V. Davidson-Shivers 
University of South Alabama, 
Alabama, USA 



Alabama. 



EDUCATIONAL ASSESSMENT VIA A WEB-BASED INTELLIGENT SYSTEM 



667 



of them are paper-based systems, which are time-consuming, error-prone, and do not scale. They are often 
unable to provide the immediate feedbacks for personalized assessment of instructional practices and, 
ultimately, improved students’ performances. 

In order to overcome these disadvantages, we propose to develop IWAS (intelligent Web-based 
assessment system), a comprehensive, real-time educational assessment system, as a common platform to 
evaluate both learning and teaching. The ultimate goal of our system is to facilitate complex learning instead of 
simple recall through automated reasoning (using BN (Bayesian Networks)) on students’ diverse knowledge of 
contents based on a revised Bloom’s Taxonomy and learning styles (according to Felder’s Model), and the 
instructor activities as well. 



Background 

A variety of approaches have been proposed for educational assessment in past decades. Researchers 
analyzed the relationships among assessments of teaching and learning (Bowden & Mar ton. 1998; Brown, 1999; 
Brown & Knight, 1994; Brown, Bull, & Pendelbury, 1997). It was found that in addition to its constructive role in 
evaluating student learning outcomes and teaching quality, assessment also provided a powerful tool to improve 
teaching and leaning effectiveness. According to Cerbin (1994), learning- centered course portfolios were 
employed to connect teaching assessment with learning assessment. A course portfolio is used not only to 
document students’ knowledge, but also to examine how different instructional practices are used to improve 
student’s learning as well. This method enhances both teaching and learning in a complementary fashion. 
Bergendahl and Tibell (2005) proposed a comprehensive assessment strategy that included input from written 
examinations, laboratory work, seminars, grant proposals and posters to enhance complex learning. The 
assessment methods, based on Bloom’s Taxonomy, have been used for concise analysis of students’ cognitive 
growth for many years (Baniulis & Reklaitis, 2002; Bergendahl & Tibell, 2005; Bloom URL, 2009; Burgess, 
2005; Choren, Blois, & Fuks, 1998; Court, Tung, Shehab, & Ashford, 2003; Oliver, Dobele, Greber, & Roberts, 
2004; Peat, 2000; Scott, 2003). For example, Oliver et al. (2004) proposed a grading system to assign a Bloom 
Rating for a course test, which indicated the cognition difficulty of that specific test. The proposed Bloom Rating 
was applied in six IT (Information Technology) courses, with the conclusion that some entry-level programming 
courses were more difficult to teach than advanced programming courses. Another example was Scott’s (2003) 
application of Bloom’s Taxonomy in computer science exams in order to verify the students’ understanding of 
course materials. 

Recent progresses in computer and Web-based technology have advanced the state of the ail significantly. As 
a result, increasing educational approaches have been transformed from traditional paper-and-pencil format to 
computerized and web-based format. For instance, Baker and Mayer (1999) analyzed the essential components of 
the computer-based assessment for students’ problem solving capability, which included problem translation and 
integration as well as solution planning and execution. A detailed survey on recent Web-based educational 
assessment systems could be found in Brusilovsky and Miller (1999). Compared with traditional paper-and-pencil 
based assessment approaches, current computer or Web-based assessment tools have not only become more 
efficient, effective and accurate, but also are able to facilitate much longer and larger application ranges. Moreover, 
Web-based systems offer many novel features that cannot be implemented in the paper-based systems, such as 
real-time data collection, management and analysis capability; individualized learning and teaching assessment; and 
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distributed and interactive assessment. Finally, increased student numbers and decreased staff resources expedite 
the need to develop electronic learning and assessment methods for students and instructors. With superior features 
offered by Web technologies, several, more recent Web-based learning systems adopted Bloom’s Taxonomy as the 
framework for individualized learner assessment (Court et al., 2003; Peat, 2000) and automatic quiz generation and 
correction (Choren et al., 1998). Additionally, a new question style that uses computer graphics to implement high 
level questions from Bloom’s Taxonomy was proposed by Baniulis and Reklaitis (2002). 

Our Solutions 

According to the analysis of problems found in existing systems, we propose IWAS to overcome the 
aforementioned disadvantages. IWAS is an intelligent, generalized, and real-time system that incorporates the 
BN technique, the client/server architecture and Web technologies. By assessing both learning and teaching 
activities, IWAS provides a foundation for educational activities leading to more effective, complex learning 
outcomes and more efficiency in instructional activities and ultimately, students’ performances. 

Intelligent Reasoning Through BN 

Research findings revealed the necessity to incorporate different learning styles into the educational 
assessment in order for a more accurate and meaningful outcome to be achieved (Ayre & Nafalski, 2000; Felder, 
1993; Felder & Silverman, 1988; Mockford & Denton, 1998). It has been made evident that both students and 
teachers prefer specific learning styles, in which they are comfortable in receiving and giving information, which 
they perceive as effective. Commonly used, Felder’s “Learning Style Model” categorizes an individual’s learning 
style along a sliding scale of four dimensions: active -reflective, sensing-intuitive, visual-verbal and 
sequential-global (Felder, 1993; Felder & Silverman, 1988). For example, the visual-verbal scale is a continuum 
in which students with high visual style prefer things that are illustrated, such as pictures, diagrams and flow 
charts, and at the opposite end, those with high verbal ability prefer written or spoken explanations, such as texts, 
print materials and written notes. Because there may be a mismatch between a student’s learning style and the 
assessment format, inaccurate assessment results may occur. Therefore, it is imperative for instructors to integrate 
individual learning styles into assessment in order to balance various assessment methods. However, it is a 
difficult matter to smoothly incorporate different learning styles into an assessment process. Most of the existing 
systems (if they even consider learning styles) adopt ad-hoc, predefined parameters that are based on human 
heuristics and/or try-and-error. Obviously, such handling of learning styles is not only error-prone, but also does 
not scale as well. In order to better model the cause -effect relationship between learning styles and learning 
outcomes, we propose to integrate BN techniques as an intelligent reasoning mechanism. 

Being a probabilistic graphical model, the BN is widely applied to automatically and accurately estimate 
the cause -effect relationship. Briefly speaking, a BN was a graph consisting of a set of nodes and arcs (Jensen, 
2007). The nodes are the random variables defined by the model to capture the task field, and the arcs encode 
the dependence relationships among the nodes. Specifically, the dependence relationships are quantitatively 
characterized by conditional probability distributions, one for each node (i.e., variable) conditional on its 
predecessor nodes. The graphical structure and the associated conditional probability tables of a BN altogether 
represent a joint probability distribution over all the variables. Given the causes (different learning styles), a BN 
can be utilized to automatically compute the probabilities of the presence of the effects (learning outcomes). In 
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this sense, a BN provides principled approaches for automatically reasoning the uncertain relationships among 
variables and, in addition, the models will be update given the observational data. 

Note that research about cognition supports the idea that learning takes place at different knowledge levels. 
As a well known theory of learning outcomes. Bloom’s Taxonomy identified six levels within the cognitive 
domain, from the simple recognition or recalling of facts as the lowest level, through increasingly more 
complex and abstract mental levels, to the highest level of evaluation (Bloom, 1956; Bloom URL, 2009; 
Burgess, 2005; Krathwohl, Bloom, & Bertram, 1973). Bloom’s Taxonomy is a precise and concise model for 
cognitive domain analysis, which is able to offer instructors a system for comprehensively evaluating students’ 
performances instead of only checking their grades. Due to the fact that students’ prior knowledge levels also 
play an important role in their learning, it is necessary to include different knowledge levels into our proposed 
BN model. Likewise, the relationship between a student’s prior knowledge level and learning outcomes can be 
handled well by BN techniques. Therefore, through enabling automated reasoning on students’ different 
knowledge levels and learning styles, IWAS is an intelligent system with least human intervention. 

Correlated Assessment on Learning and Teaching 

The absence of teaching practice assessment has been an issue neglected in most of current educational 
assessment systems. Although teaching assessment is conducted through course evaluations by students, peer 
and department chair annual reviews, and tenure and promotion committees, it may not be beneficial because it 
is delivered in a summative fashion. A more optimal approach would be through a series of convenient and 
instant formative assessment, which allows instructors to monitor and boost teaching practices. 

In the IWAS system, the instructor assessment is addressed via the comprehensive feedbacks provided 
from three different levels. They are as follows: 

The low level students’ evaluation. A variety of statistics of student learning assessment results are 
provided by the system, such as test scores and rankings, knowledge levels in which students fail and succeed, 
learning styles and preferences reported by students, questionnaires about students’ opinions about questions 
and tests, etc.. According to the instructors’ requirements, these data can be collected in actual documents (i.e., 
class inquiries, quizzes and lab assignments), and in real time (i.e., daily, weekly, monthly or by semester). An 
instructor can easily conduct a self-evaluation through these data by investigating students’ performance at 
specific knowledge levels and topics, as well as by comparing with previous or other schools’ student works on 
the same questions. This type of feedbacks allows instructors to trace closely the progress of each student and 
revise daily teaching plans for effective intervention. For example, ignoring well understood topics, repeating 
questions of suitable level and formatting for individual students until they master the principles and enhancing 
their cognition levels and so on. Additionally, like other course management systems, i.e., Angel Learning 
(2009), Blackboard (2009), CourseCompass (2009), Moodel (2009) and QuizStar (2009), IWAS also provides 
an informative tool for instructors to deliver course contents adaptive to individual student’s learning styles. 

The middle level peer reviews. An instructor’s colleagues can adapt the instructor’s assessment materials 
to their own classes. Through colleagues’ reports, the instructor receives peer evaluation on his/her instructional 
practices, and assessment strategies and materials. For example, suitability and correctness on question 
description, level categorization, and learning style adaptation, etc.. Based on the peer review feedbacks, an 
instructor is able to learn new teaching and assessment strategies and materials from colleagues, thus enhancing 
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his/her own strengths and overcoming weaknesses in future teaching and assessment. 

The high level administrative evaluations. An instructor’s supervisors during semesters (i.e., department 
head, faculty evaluation committee) can check the appropriateness of the instructor’s course contents and 
teaching and assessment strategies according to department and university policies and mission. With these 
formative feedbacks, an instructor will be enabled to modify strategies and contents for teaching and 
assessment to meet the department and/or university requirements before the end of academic years. Note that 
this information is also useful for external accreditation review. 

By comprehensively assessing instructor activities, IWAS aims to correlate the teaching assessment with 
various learning assessments, with the purpose of improved efficiency and effectiveness in instructional activities. 

Enhanced Generalization via Modularized Architecture 

The lack of generality for broader purposes is another common disadvantage of the majority of existing 
educational assessment systems. That is, they are usually applications or domains specific, and very few 
provide a general model for a systematic learning assessment. As a result, these systems have limited value and 
cannot be generalized for broader purposes. 

In order to obtain an enhanced generalization, IWAS is developed under a client/server architecture, which 
consists of two different types of components: (1) the server component that acts as the repository for data 
collection and the provider for system services; and (2) the client component that serves for human-computer 
interfaces. In addition, the system is implemented as a modularized model, i.e., IWAS is decomposed into a set 
of independent modules with different objectives and functionalities. By adopting an open system architecture 
and the standard interfaces among different modules, it is not difficult for outdated modules to be enhanced or 
simply removed, and new modules can be readily plugged in as well. Such flexibility in maintaining and 
upgrading the system makes IWAS a generalized system that will be adaptable to many different domains. The 
next section provides greater details on the overall system architecture and different modules. 

Real-Time Feedback Based on Web Technologies 

Homework, quizzes, tests, projects and final exams supply the usual components of learning assessment 
system. By the end of a semester, students are expected to understand all the required course contents and the 
master essential skills. However, there are many factors that have an effect on student learning outcomes. More 
often than not, the assessment results and the causes for poor students’ performances are not delivered in a 
timely manner. As a result, students who do not understand the materials in a class may become confused and 
unable to relay this information to the instructor in time. Therefore, a real-time assessment strategy needs to be 
defined and implemented for both learning and teaching. 

IWAS utilizes up-to-date computer and Internet technologies, along with cutting-edge Web technologies, 
so that the formative feedbacks can be provided to the users in a timely manner (essentially with an instant 
timeframe). In this way, both learning outcomes and the instructional practices will be further enhanced. 

System Architecture and Implementation 

In this section, we are going to introduce the overall architecture of our proposed IWAS system. We will 
then describe different modules contained in each subsystem, and provide detailed information on system 
implementation. Finally, preliminary experiment results will be reported. 
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Overall System Architecture 

As illustrated in Figure 1, IWAS is based on a client/server architecture design. The framework consists of 
two major components: (1) the central server component that acts as the repository for data collection and the 
system services provider; and (2) the client component (e.g., laptops, desktop computers, Pocket PCs or cell 
phones) that serves for human-computer interfaces. 




Windows Server 2003 

Web Server (IIS, ASP.NET) 



Administrator, Instructor 
& Student Subsystems 



YB.NET 
JavaScript 
ADOAEL^ 



Figure 1. The overall architecture of IWAS system. 



The system functional component diagram is demonstrated in Figure 2. As shown in the figure, the system 
consists of three subsystems: (1) the AS (Administrator Subsystem); (2) the IS (Instructor Subsystem); and (3) the 
SS (Student Subsystem). All these three subsystems are based on a central DC (Database Component), and they 
all include a group of system modules. Different subsystems provide the system interfaces for different users. 




Figure 2. The functional components diagram of IWAS system. 



Modules in Each Subsystem 

The Administrator Subsystem contains three modules, i.e., “Instructor and Student”, “Question and Test” 
and “System”; the Instructor Subsystem has two modules, i.e., “Instructor Portfolio” and “Test Creation”; and 
the Student Subsystem contains five modules, i.e., “Student Portfolio”, “Test Selection”, “Self Assessment”, 
“Test Taking” and “Grading”. In addition, there arc five other modules that are common to all these three 
subsystems, i.e., “Login”, “Help”, “Query”, “Communication” and “User Report”. 
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Table 1 



Functionalities of Each Module in IWAS System 



Module name 


Major functionality 


Subsystem 


Al: Instructor 

and student 


Instructor and student account management: account configuration; adding/deleting/updating 
accounts; and profile management. 




A2: Question and 
test 


Question and test management: question categorization by topic, taxonomy level, author, 
created time, used times and targeted learning style; logic-based configuration for automatic 
test status categorization; expired question and test removal; and pre/post-test 
survey/questionnaire development and delivery. 


Administrator 

subsystem 


A3: System 


System maintenance: system usage and performance surveillance; system log file audit and 
query; system configuration and update backup; and periodic system report production, e.g., 
system usage and maintenance. 




11 Instructor 
portfolio 


Teaching portfolio management: previous tests tracing; current student supervision; student 
learning style classification and learning curve identification; and customized test 
categorization and delivery. 


Instructor 

subsystem 


12: Test creation 


Test creation: test development and query; learning style-based question development; 
intelligent question taxonomy level identification; individualized test creation; 
norm-referenced or criterion-referenced grading scheme design; and configuration for student 
access to test solutions. 


SLStudent 

portfolio 


Learning progress records: previous courses, tests and question tracing; intelligent 
self-practice; and learning style self-report. 




S2: Test selection 


Required tests selection: test assignment collection and selection; and test query by deadline, 
time duration, topic, coverage and pre-requisite. 




S3: 

Self-assessment 


Self -practice recommended by instructors: self-practice tests selection; question and test query 
by topic, pre-requisite, system and instructor recommendation, and learning style and 
knowledge level; adaptive annotations on each question about topic, learning style, knowledge 
level, and goal relevancy; and automatic question/test delivery according to topic, knowledge 
level and learning styles. 


Student 

subsystem 


S4: Test taking 


Test delivery: categorized and customized question presentation by taxonomy level and 
learning style. 




S5: Grading 


Score computation: automatic question weight assignment and test score derivation; 
semiautomatic essay grading; and test results recording. 




Cl: Login 


User login and logout: new account construction; user identification and categorization; and 
system access configuration. 




C2: Help 


System handbook: static help files (HTML, XML orXSL); and animated tutorial (flash/video). 




C3: Query 


User request response: instructor and student query; and test/question query. 


Common 


C4: 

Communication 


User communication: email; bulletin board system; and chat room. 




C5: User report 


Report generation: test and question result statistic analysis; user report generation; test and 
question report generation; survey; and questionnaire. 





Implementation Details 

The server runs on Windows Server 2003 operating system, hosting server functionalities such as Web 
service and database service. Web server is built with Microsoft IIS (Internet information service) and 
ASP.NET (Active Server Page.NET). In the system, the current version of IIS 6.0 is adopted as the Web server 
to provide a highly reliable, manageable, and scalable Web application infrastructure. ASP.NET 2.0 is a set of 
powerful Web application development technologies comprising the Microsoft.NET framework. Three 
subsystems are programmed by JavaScript and Microsoft Visual Basic.NET 2005. The system user information 
and statistical data of tests are stored and managed by the database management system SQL sever 2005. In the 
subsystem programs, we use the classes of Microsoft ADO.NET (ActiveX Data Object.NET) to retrieve and 
update the database. Crystal Report is adopted to produce different reports. To secure the data transmission, we 
use https instead of http, and clients will be able to communicate with the SQL Server through IIS with SSL 
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(secure sockets layer). 

Besides the synchronous mode for online system access and data analysis, the IWAS system also supports 
asynchronous mode by providing capabilities for offline data collection and processing, i.e., faculty and 
students can download tests or reports to local disks for analysis or presentation. The system provides the 
functionality to export data into Microsoft Word/Rtf/Excel/ Access format. 

Preliminary Experiments 

The IWAS system has been developed and tested in science courses of a local high school. We selected 
three biology courses from three different levels (regular, advanced and advanced placement) for system testing. 
There were five high school teachers and 23 students participating in the experiment. The teachers have created 
more than 400 questions in the system database. According to the questionnaires and surveys, all the instructors 
and more than half of the students in the present study thought that IWAS was more challenging, helpful and 
beneficial than traditional paper-based approaches. To be more specific, the students commented that the best 
aspects of the system were the immediate feedbacks, the clarity of materials, the anonymous ranking and 
comparison, the ease of use, and the great availability (any time, any place and any pace). At the same time, the 
teachers enjoyed the easy-to-use test creation, assignment and grading functions, the personalized assessment, 
and the instant feedbacks and analysis. The system allows teachers to trace closely the progress of each student 
and revise their daily teaching plans for effective intervention. Analysis of user responses from our preliminary 
experiments has also given useful information for future improvements. The complaints have concentrated on 
technological problems, and most were to do with hardware problems, e.g., slow CPU (central processing unit), 
retarded network transfer and insufficient memory. These problems may be addressed with additional 
information online for students about the technical requirements of the system. 

Conclusions and the Future Work 

In this paper, we propose IWAS, which is an intelligent, generalized and real-time system incorporating 
the BN technique, the client/server architecture and Web technologies. By assessing both the learning and 
teaching activities, IWAS provides a foundation for educational activities leading to more effective, complex 
learning outcomes, i.e., more efficiency in instructional activities, and ultimately, students’ performances. 
Preliminary experiment results have demonstrated the promising performance of the proposed system. In the 
future, we plan to expand the system to support more functional modules; we will also disseminate it to both 
K-12 schools and colleges. 
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